fix: add missing index field to streaming tool_call chunks#49
Merged
CaddyGlow merged 2 commits intoCaddyGlow:mainfrom Mar 22, 2026
Merged
Conversation
The OpenAI Chat Completions streaming API requires each tool_call delta to include an `index` field that identifies which tool call in the array the chunk belongs to. Without it, clients like the genai Rust crate silently discard tool_call chunks, causing agents to terminate early without executing any tools. Changes: - Add optional `index` field to `ToolCall` model - Pass `index` through `build_openai_tool_call` helper - Set `index=state.index` on all `ToolCall` instantiations in openai_to_openai and anthropic_to_openai streaming formatters - Set `index` in claude_sdk parser tool call conversion
The OpenAI spec defines two distinct types for tool calls: - ChatCompletionMessageToolCall (non-streaming): id, type, function - ChoiceDeltaToolCall (streaming): index (required), id, function, type Split the single ToolCall model into ToolCall (non-streaming, no index) and ToolCallChunk (streaming, required index). This keeps non-streaming responses spec-compliant while enforcing index on streaming chunks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The OpenAI Chat Completions streaming API spec requires each
tool_callsdelta chunk to include anindexfield (ref). This field identifies which tool call in the array the chunk belongs to, which is essential when the model invokes multiple tools in parallel.Currently, ccproxy omits
indexfrom streamingtool_callschunks. This causes clients that rely onindexfor chunk reassembly (e.g., the genai Rust crate) to silently discard all tool call data, resulting in agents that terminate immediately without executing any tools.Root Cause
The
ToolCallPydantic model inccproxy/llms/models/openai.pylacked anindexfield. Streaming formatters that constructToolCallinstances (anthropic_to_openai, openai_to_openai, claude_sdk) never set it, so serialized chunks were missing the field entirely.Changes
ccproxy/llms/models/openai.py: Addindex: int | None = NonetoToolCallccproxy/llms/formatters/anthropic_to_openai/_helpers.py: Accept and pass throughindexinbuild_openai_tool_callccproxy/llms/formatters/anthropic_to_openai/streams.py: Pass accumulatedindexwhen building tool callsccproxy/llms/formatters/openai_to_openai/streams.py: Setindex=state.indexon allToolCallinstantiationsccproxy/llms/formatters/openai_to_openai/responses.py: Setindexbased on position in tool_calls listccproxy/plugins/claude_sdk/parser.py: Pass positionalindextoformat_openai_tool_callBefore (broken)
{"delta": {"tool_calls": [{"id": "fc_xxx", "type": "function", "function": {"name": "list_files", "arguments": ""}}]}}After (fixed)
{"delta": {"tool_calls": [{"index": 0, "id": "fc_xxx", "type": "function", "function": {"name": "list_files", "arguments": ""}}]}}Impact
This fix restores compatibility with any OpenAI-compatible client that parses streaming tool calls per the API spec. Without it, tool-calling agents built on frameworks like tirea + genai cannot function through ccproxy.